-
Notifications
You must be signed in to change notification settings - Fork 119
Change host's memory setup to be minimally configured #297
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
danbugs
wants to merge
26
commits into
hyperlight-dev:main
Choose a base branch
from
danbugs:danbugs/memory-layout
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
danbugs
added a commit
to danbugs/hyperlight
that referenced
this pull request
Feb 27, 2025
…st code to start at 0x0 instead of after paging sections This is the first step in moving memory sections around to provide a more generic layout. After this, I'll move continue moving sections to fulfill the layout shown in the description for PR hyperlight-dev#297. Changes: - removed no longer applicable checks in hypervisor_handler - changed the way we build memory in get_memory_regions - added functions to get the offsets/addresses to paging sections as they are no longer static from 0x0 and are now relative to code size - updated code docs - changed set_up_shared_memory to properly place paging sections - updated paging docs Signed-off-by: danbugs <[email protected]>
danbugs
added a commit
to danbugs/hyperlight
that referenced
this pull request
Feb 27, 2025
…st code to start at 0x0 instead of after paging sections This is the first step in moving memory sections around to provide a more generic layout. After this, I'll move continue moving sections to fulfill the layout shown in the description for PR hyperlight-dev#297. Changes: - removed no longer applicable checks in hypervisor_handler - changed the way we build memory in get_memory_regions - added functions to get the offsets/addresses to paging sections as they are no longer static from 0x0 and are now relative to code size - updated code docs - changed set_up_shared_memory to properly place paging sections - updated paging docs Signed-off-by: danbugs <[email protected]>
56852a6
to
40a8091
Compare
danbugs
added a commit
to danbugs/hyperlight
that referenced
this pull request
Feb 28, 2025
…st code to start at 0x0 instead of after paging sections This is the first step in moving memory sections around to provide a more generic layout. After this, I'll move continue moving sections to fulfill the layout shown in the description for PR hyperlight-dev#297. Changes: - removed no longer applicable checks in hypervisor_handler - changed the way we build memory in get_memory_regions - added functions to get the offsets/addresses to paging sections as they are no longer static from 0x0 and are now relative to code size - updated code docs - changed set_up_shared_memory to properly place paging sections - updated paging docs Signed-off-by: danbugs <[email protected]>
b7435ae
to
ce255f8
Compare
danbugs
added a commit
to danbugs/hyperlight
that referenced
this pull request
Mar 17, 2025
…st code to start at 0x0 instead of after paging sections This is the first step in moving memory sections around to provide a more generic layout. After this, I'll move continue moving sections to fulfill the layout shown in the description for PR hyperlight-dev#297. Changes: - removed no longer applicable checks in hypervisor_handler - changed the way we build memory in get_memory_regions - added functions to get the offsets/addresses to paging sections as they are no longer static from 0x0 and are now relative to code size - updated code docs - changed set_up_shared_memory to properly place paging sections - updated paging docs Signed-off-by: danbugs <[email protected]>
8aa7f89
to
f21aabc
Compare
danbugs
added a commit
to danbugs/hyperlight
that referenced
this pull request
Mar 17, 2025
…st code to start at 0x0 instead of after paging sections This is the first step in moving memory sections around to provide a more generic layout. After this, I'll move continue moving sections to fulfill the layout shown in the description for PR hyperlight-dev#297. Changes: - removed no longer applicable checks in hypervisor_handler - changed the way we build memory in get_memory_regions - added functions to get the offsets/addresses to paging sections as they are no longer static from 0x0 and are now relative to code size - updated code docs - changed set_up_shared_memory to properly place paging sections - updated paging docs Signed-off-by: danbugs <[email protected]>
f21aabc
to
53069f2
Compare
danbugs
added a commit
to danbugs/hyperlight
that referenced
this pull request
Mar 19, 2025
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and remove statically defined memory regions. This commit updates documentation in accordance to these changes. Signed-off-by: danbugs <[email protected]>
4980140
to
3212be7
Compare
danbugs
added a commit
to danbugs/hyperlight
that referenced
this pull request
Mar 19, 2025
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and remove statically defined memory regions. This commit updates documentation in accordance to these changes. Signed-off-by: danbugs <[email protected]>
3212be7
to
be061f0
Compare
danbugs
added a commit
to danbugs/hyperlight
that referenced
this pull request
Mar 19, 2025
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and remove statically defined memory regions. This commit updates documentation in accordance to these changes. Signed-off-by: danbugs <[email protected]>
be061f0
to
76d67a8
Compare
danbugs
added a commit
to danbugs/hyperlight
that referenced
this pull request
Apr 11, 2025
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and remove statically defined memory regions. This commit updates documentation in accordance to these changes. Signed-off-by: danbugs <[email protected]>
06dd5e3
to
6e82f64
Compare
danbugs
added a commit
to danbugs/hyperlight
that referenced
this pull request
Apr 14, 2025
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and remove statically defined memory regions. This commit updates documentation in accordance to these changes. Signed-off-by: danbugs <[email protected]>
cbf1071
to
63e3c69
Compare
Signed-off-by: danbugs <[email protected]>
In hyperlight-dev#297, we make the base address of guest memory 0x0 instead of 0x200_000 and remove statically defined memory regions. This commit updates documentation in accordance to these changes. Signed-off-by: danbugs <[email protected]>
…yout + removes MemMgrWrapper This commit completely removes the file w/ the mem layout. There are still references to it around, but those will be removed in subsequent commits. With the upcoming refactors, there is no need for the MemMgrWrapper abstraction. This commit removes in in favour of directly utilizing a SandboxMemoryManager<S> in subsequent commits. Signed-off-by: danbugs <[email protected]>
- removes old HyperlightPEB structures (making it more idiomatic Rust). - removes input/output structures from guest library, moving it to common as they will need to be used by external guests. - creates new HyperlightPEB API w/ capabilities to address memory regions from the guest. Signed-off-by: danbugs <[email protected]>
…uilder This commit partially introduces the SandboxBuilder mentioned on hyperlight-dev#249, changing the way we create uninitialized sandboxes. - adds SandboxBuilder API, which allows for a more flexible memory setup. - changes to uninit w/ how it is created. - deletes memory_region.rs file w/ most of it being superseded by the SandboxBuilder structures. - simplifies config.rs removing all items related to previous static memory layout. Signed-off-by: danbugs <[email protected]>
… in common library and use CGM The host now defines a single undifferentiated memory region—the custom guest memory region (or, CGM)—that the guest can address as it sees fit. This commit changes the guest library to leverage that and perform said address using the APIs defined in the common library. Signed-off-by: danbugs <[email protected]>
- removes static layout from SandboxMemoryManager, adds SandboxMemorySections structure built w/ the SandboxBuilder. - updates paging to reflect usage of dynamic memory layout. - interact with common APIs. - add APIs to allow reading addressed memory sections in PEB from shared memory. Signed-off-by: danbugs <[email protected]>
… codebase - temporarily comment out a lot of code (e.g., host/testing/mod.rs). - temporarily unuse seccomp stuff. - start using new mgr API and common API. - pending removal of writting host detail functions to memory. - change hv_handler to use new mgr API to get guest_function_dispatch_ptr. - remove layout/memory_region from mod.rs. - change RawPtr API (should probably be re-evaluated). Signed-off-by: danbugs <[email protected]>
Updates drivers to map memory region in accordance to CGM usage. > Note: only updated KVM fully so far. WHP and inprocess drivers are TODOs. Signed-off-by: danbugs <[email protected]>
Signed-off-by: danbugs <[email protected]>
…d a DebugPrint OutBAction - the mechanism for causing VM exits and its abstractions over host functions should be exposed in the common library so they can be used externally. - 'PrintOutput' should not be a special host function. Instead, we should provide functionality for debug prints—i.e., the DebugPrint OutBAction. Signed-off-by: danbugs <[email protected]>
If there's a mismatch in the guest's usize and the host's usize, our input and output stacks would break. This commit fixes that by using u64 instead of usize for ptr accesses. Signed-off-by: danbugs <[email protected]>
Removed an uneeded entrypoint argument for the size of the PEB structure, which was a re-use of the ops argument in dev that is also unused. + cleaned up some TODOs (better error messages for unwraps and uneeded host fxn details section). Signed-off-by: danbugs <[email protected]>
(1) had to add helper impl functions for the HyperlightPEB struct to get the host addresses for some memory regions. (2) re-added some logic for the LeakedOutBWrapper and for the InProcessDriver. (3) added an in process test in the sandbox_builder file. (4) renamed hyperlight_common::hyperlight_peb mod to hyperlight_common::peb. Signed-off-by: danbugs <[email protected]>
Needed to make some modications to hyperv_linux for it to conform to the new SandboxMemorySections API. With that, I also had to add a From impl to sandbox_builder. As an aside, I also modified the signature of get_memory_access_violation to take a borrow of SandboxMemorySections instead of an owned value, which ended up impacting kvm too. Signed-off-by: danbugs <[email protected]>
Changed whp driver to use the new SandboxMemorySections API. Signed-off-by: danbugs <[email protected]>
…shdump feature Crashdump feature used the old MemoryRegion API. Updated it to use the new SandboxMemorySections API (causing changes in all drivers) and added a test in sandbox_builder for the feature. Signed-off-by: danbugs <[email protected]>
- cleaned up the HyperlightPEB API in the common library. -- added MemoryRegion struct to better group related offsets and sizes. -- removed pub fields from HyperlightPEB struct making fields accessible only via getters/setters. -- cleaned up, commented, and re-organized existing fxns for HyperlightPEB struct. - now we modify the rsp in the host if the guest sets up a new stack region (i.e., essentially dropping the tmp stack). Signed-off-by: danbugs <[email protected]>
- the guest error data region was removed and now, instead, we leverage the output data region to communicate guest errors back to the host. - previously, the host error data region was incorrectly accessed making it essentially unused. This commit removes the region altogether. There is some merit to the region existing in some way, but this is outside of the scope of hyperlight-dev#297. Signed-off-by: danbugs <[email protected]>
- updated our hello-world example, - updated the README, - changed last remaining use of HostPrint to use print (wrapper for using the DebugPrint OutbAction), and - added an extra test in sandbox_builder. Signed-off-by: danbugs <[email protected]>
…dler + re-added seccomp test Previously set mem_access_handler_fn to be NOOP. Brought back original functionality; albeit a bit questionable. Signed-off-by: danbugs <[email protected]>
d126ecc
to
846780a
Compare
Tests are already covered in the sandbox_builder. In 75b0076, I also added a test for Windows in-process mode using the load library. Signed-off-by: danbugs <[email protected]>
- Changed the ABI for the inprocess outb handler from `extern "C"` to `extern sysv64`. This was needed because `extern "C"` caused issues on Windows. Previously, we used `extern "win64"`, which was problematic for guests intending to use `hyperlight_common` and not supporting the Windows ABI. - Added a test for running inprocess on Windows w/ load library. Signed-off-by: danbugs <[email protected]>
75b0076
to
04e8e96
Compare
Signed-off-by: danbugs <[email protected]>
3c6255b
to
815cb5b
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We want to allow embedding custom guests in Hyperlight, but, currently, the static memory layout we configure in the host is too strict. This PR is modifying our memory layout to be minimally configured.
Here's an example of how a guest could address its memory:

Changes:
SandboxBuilder
API.MemMgrWrapper
abstraction.TODO:
OutBAction::DebugPrint
instead of HostPrint host fxn,